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ABSTRACT 


This report describes human performance measurement techniques for 
remote manipulation tasks and remote sensing techniques for manipulators. 
For common manipulation tasks, performance is monitored by means of an 
on-line computer capable of measuring the joint angles of both master and 
slave arms as a function of time. The computer programs allow measure- 
ments of the operator's strategy and physical quantities such as task time 
and power consumed. The results are printed out after a test run to com- 
pare different experimental conditions. 

For tracking tasks, we describe a method of displaying errors in 
three dimensions and measuring the end-effector position in three 
dimensions . 

We also report progress on remote sensing systems based both on touch 
and distance sensing. The touch-sensing system uses proportional force 
sensors distributed over the remote hand to measure the overall force 
distribution of objects against the hand. The range sensor uses reflec- 
tion from infrared light beams to identify the position of objects at 
a distance of a few centimeters to more than 20 centimeters in front of 
the hand. 
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I INTRODUCTION 


This report describes continued progress toward the development of 
(1) measurement techniques and tasks for evaluating man^s manipulative 
performance and (2) remote sensing and display techniques to augment man^s 
manipulative skills. Much of the work is a further development of the 
manipulator system previously described,^* 

To study human performance both in the time delay situation and 
situations using different displays^ manipulators, and controls, the 
pilot experiments of Sections II and III were carried out. The pilot 
experiments provided a data bank for the logging and analysis programs 
developed in Section V; the new data analysis techniques are being tested 
out and debugged. Section V describes the experimental configuration for 
carrying out both compensatory and pursuit tracking tasks in space. With 
the multicoordinate tracking program previously described,^ it will be 
possible to measure describing function remnant spectra and root-^mean- 
square (rms) tracking errors. 

For automatic control of remote manipulators and to enable man to 
have better manipulative skills, certain features of the remote environ- 
ment must be sensed. Sections VI and VII describe the continued develop- 
ment of sensors for this purpose. Development of the touch sensing sys- 
tem begun under NASA-SNSO support^ is being continued here. The goal is 
to develop a touch sensing and display system that gives man the ’^feel'’ 
of the remote objects he is handling. 


* 

References are listed on page 32 of this report. 
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II PILOT TACTILE DISPLAY EVALUATION 


Experiment I approximates the experiment described in Appendix F 
of Reference The only modification of the experiment was to perform 

eight repeated pick-up attempts instead of ten. Since performance seemed 
to be improving as the experiment progressed, the entire experiment was 
repeated a second time in reverse order. We hope that the additional 
data can be used to plot learning curves and that both learning rates and 
ultimate performance levels can be used to differentiate between experi- 
mental conditions. 

Performance measurements consist of both the task time for each run 
as measured by a stopwatch and the number of fumbles or drops occurring 
during each experimental condition. In all conditions except the visual 
sensor display, the performance monitor package described in Section IV 
of this report was used. 

At the present time the data from the experiment are being reduced 
and prepared for plotting, analyses of variance, and other statistical 
tests. 
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Ill PILOT Tim DELAY EXPERIMENT 


A, General 

This experiment investigates changes in the operator's manipulation 
strategy as the transmission delay is varied over a wide range. The 
approach is to obtain a bank of human performance data using the data- 
logger described in Section IV of this report and then to experiment 
with several performance indices to find those that best characterize 
the changes in strategy from a continuous strategy to a move-and-wai t 
strategy as the time delay increases from zero to well over one second. 
The areas of particular interest are numbers of moves, relations of arm 
motion to decisions to move, and descriptions of the complex move-and- 
wait strategy. The measurements made by the off-line analyses of Sec- 
tion IV represent an initial attempt to obtain such indices. The results 
of this pilot time-delay experiment will probably suggest other still 
more pertinent manipulation parameters to be obtained while an experiment 
is being carried out. 

B. Method 

The experiment is arranged in a 3 x 5 x 2 factorial design, as shown 
in Figure 1. Each cell in the design represents a performance charac- 
teristic measured on two subjects in eleven repetitions of the task. 

The experiment variables are (1) manual control mode, (2) trans- 
mission delay, and (3) replication, as indicated in Figure 1. The manual 
control mode Is varied by use of either the Rancho master brace or a 
bank of six potentiometers. Transmission delays from zero to ten seconds 
are provided in both control conditions, and in all replications by using 
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SA-1587-33R 

FIGURE 1 DESIGN OF THE PILOT TIME DELAY EXPERIMENT 

the 30-Hz delay line simulation described in Section II of Reference 1. 
Direct viewing was used and audio cues were provided in all experimental 
cases . 

1 * Apparatus 

The Rancho arm and computer augmented control system described 
in Section II of Reference 1 were used for this experiment. The control 
modes were solely manual, master-slave modes. No sensory feedback other 
than direct vision was provided to the operator. The task was to pick 
up a block randomly placed within the arm workspace and deposit it in a 
small container, 
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2 . 


Subjects 


Two male subjects, LM and SM, were used for this experiment. 
Both had considerable experience in using the manual control modes for a 
pick-up task. However, neither subject had ever attempted the task with 
a transmission delay. 

3, Procedure 

The on-line performance logger (see Section IV) is started when 
the end effector passes through a plane one-foot above the table on the 
way down to grasp the object. The task is complete when the object is 
grasped and deposited in the receptacle, and the end effector moves up 
out of the workspace. Then the performance logger stops. 

In a single replication, each subject performed 10 runs consisting 
of 11 repetitions each. Five runs, each corresponding to one of the 
transmission delays, were performed using each of the two control modes. 
This sequence was repeated three times for each subject. In all, each 
subject made 333 individual pick-up attempts. 

C. Analyses Underway 

1 . Performance Measures 

A5x3x2x2x 11 analysis of variance of each of the per- 
formance indices determined by the performance monitor (Section IV) is 
being carried out to determine their relative importance in differentiat- 
ing between test conditions. 

2. Movement Time Distribution 

We plan Chi-square analyses to determine whether the movement 
time distribution (Section IV) changes significantly with varying trans- 
mission delay and control Input. 
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IV EXPERIMENT ANALYSIS TOOLS 


A performance monitor package was created to study (1) the complex 
move~and— wait strategy and (2) the movement and waiting times with dif~ 
ferent transmission delays, different visual and tactile feedback, and 
different arms. The performance monitor can measure and tabulate the 
movement and waiting times with considerably greater accuracy and reli- 
ability than can a human observer with a stopwatch. 

The performance monitor package consists of an on-line program for 
data logging and several off-line programs for numerical analysis. During 
the experimental runs, a high-speed disk memory logs on-line data. Upon 
completion of the experiment, it is copied to magnetic tape for permanent 
storage. Different off-line programs are used to search the log and to 
extract the desired performance indices. This section gives the descrip- 
tion of the components of this performance measuring system. 

A. Using the On-Line Performance Logger 

The control codes available to the experimenter for accumulating 
and logging data are shown in the control tree of Figure 2, A typical 
control sequence for logging two replications of an experiment is shown 
in Figure 3, where information typed by the operator is underlined. 

B. Operation of the On-Line Performance Logger 

The on-line performance logger detects the beginning and end of moves 
by using derivatives of the individual Joint angles. In total, 14 

An alternative method of detecting moves based on a calculation of 
instantaneous joint velocity is being investigated. 
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SA-2 583-1 


FIGURE 2 COMMAND STRUCTURE FOR THE ON-LINE 
PERFORMANCE LOGGER 


t OPEN * PFILE 
ENTER NAME, DATE 
t SUBJECT L 6/10/72 TEST 2 
t START 
t FINISH 
t START 
t KILL 
FINISH 
t MESSAGE 

* BLOCK FELL TO GROUND, TRY AGAIN 
t START 
t FINISH 
t CLOSE 
t QUIT 


Logging In 


Replication 1 


Replication 2 Unsatisfactory 


Replication 2 o,k. 


Logging Out 


SA-2583-2 


FIGURE 3 TYPICAL PERFORMANCE LOGGER CONTROL SEQUENCE 
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derivatives (seven master and seven slave Joint angles) are updated and 
digitally filtered every 1/30 of a second. If any of the master or slave 
joints exceeds a predetermined threshold for motion during a 1/30-second 
period, a note of the fact is made in separate master and slave move 
detection queues. These queues (software shift registers) record whether 
or not a move was detected during 12 successive 1/30-second intervals. 

From these intermediate data, decisions are made to determine whether a 
master or slave move has begun or ended. The flow chart of the performance 
monitor is given in Figure 4, The criteria for detecting the beginnings 
and ends of moves that have proved successful are defined below: 

• Move criterion — A move begins when the velocity threshold is ex- 
ceeded during the current 1/30-second interval and will be exceeded 
on five of the next 12 intervals, 

• Done criterion — A move is done when the velocity threshold is not 
exceeded during the current interval and will not be exceeded more 
than once in the next 12 intervals. 

Two total task measurements are also obtained. The on-line program 
counts the number of 1/30-second intervals taken to complete a task and 
logs the total at the end to permit the calculation of task duration. 
Additionally, it accumulates the current delivered by the 24-volt servo 
power supply every 1/30 second and logs the total at the end of the run to 
permit calculation of the total energy consumed. The first three numbers 
following the 'V symbol are the triple precision accumulation of the 
current, and the next two are the double precision accumulation of the 
task time. 

Messages entered during the run are printed directly in the data log. 
Entering the *^kill’^ message during a run causes a symbol to print out 

on the data log and further logging to cease. The meanings of the various 
symbols used in the data log are given in Table 1, An example of the data 
log for run Number 1 of Subject SM is given in Figure 5. The first number 
and symbol, 0036 >, represents the one-second time delay in thirtieths of 
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FIGURE 4 


PERFORMANCE LOGGER MOVE DETECTION ALGORITHM 















Table 1 


SYMBOLS USED BY ON-LINE PERFORMANCE LOGGER 


Symbol 

Meaning 

$ 

Master move detected, counter == 0 


Master move ended 

@ 

Automatic move initiated, counter = 0 

t 

Slave move detected 

• 

Slave move ended 

’^carriage return" 

Counter before automatic or manual move begun 

> 

Time delay 

/ 

Power and time printout follow 

7 

Kill feature initiated; ignore data from 
this run 

+ 

Identifying message follows 

"end of record” 

End of replication 

"end of file” 

End of run 


♦ SUBJECT Sn BL0O< #tCKUB AND DM^ 

0036>0O36 

0000$0043toos£f0l 10*01 1 1 

0000s0020#0043t006 3* 007 3 
0000$0017#0043 

0000s0000t00£4*0043t004«i0077 

OOOOSOO 1 1 • 00£7 #0 104 t 0 I 40 • 0 1 S7 

0000SO043*O£S4«03l4 

0 OOOSOOO I • 003440042 1 0 1 00* 0 1 S4 

000OSOO1740044 

0000so000t0020#0046 

0000S0004* 002440045 f 0060 

0000s0010*0040t004240i01*/0000 0014 2277 0000 1626 


FIGURE 5 EXAMPLE OF DATA LOG 
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a second (in octal). On the last line, the first three numbers after 
the are the accumulated current, and the next two are the accumulated 

run time in thirtieths of a second (octal) ^ 

C. Off-Line Timing Program 

A computer program for obtaining particular performance figures from 
the data log has been developed. The indices obtained and the means of 
obtaining each of them is given in Table 2 . 


Table 2 

PERFORMANCE INDICES 


Symbol 

Definition 

Method of Obtaining Measurement 

N 

m 

Number of 

Counting the number of symbols 

master moves 


N 

g 

Number of 

Counting the number of ” symbols 


slave moves 


E 

t 

Task energy 

V 

times the current accumulator 

Rate 

T 

t 

Task time 

1 

times clock accumulator 

Rate 

T 

m 

Total moving 
time 

1 

„ ^ times sum of master move times; the 

Rate 

master move time for each move precedes the 
symbol. 

R 

m 

Moving ratio 

T divided by T 
m t 

M 

m 

Mean movement 
time 

T divided by N 
m m 
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The printout from the timing program for two subjects is shown in 
Figure 6* Each of the data files analyzed consists of 11 replications 
of a pickup and drop task carried out with a time delay of one second* 

The performance indices are printed out in the same order as they are 
defined in Table 2, Times are in seconds and energy is in kilowatt seconds. 

D. Off-Line Histogram Program 

In order to investigate changes in the operator's strategy under 
experimental conditions such as time delays, we wrote a program to obtain 
the distribution of move times. The algorithm is essentially that of the 
well-known ’^ul se-height analyzer*” 

In the move-time analyzer there are 51 bins for accumulating counts. 
When a master move (indicated by a in the data log) is found, the 

appropriate bin is incremented by one to count the move. The first bin 
Is for move durations of 1/30, 2/30, and 3/30 second (0.03 to 0.1 second); 
the second bin is for durations of 4/30, 5/30, and 6/30 second (.13 to 
.2 second); and so the bins continue to the highest bin which accumulates 
all moves greater than 150/30 seconds (5 seconds). After all the desired 
data logs are searched, a printout of the bins can be requested. 

An example of using the program and its resultant output is shown 
in Figure 7, Here the data analyzed are the same as those of Figure 6. 

As Figure 7 Indicates, there were 22 test runs and 447 master moves; the 
mean value of the ensuing distribution is 1.257 second. Following the 
totals is printout of the bin totals. The first number on each line is 
the bin count, and the second is the lower bound of each bin in seconds. 

The bin counts are illustrated graphically by printing one space for each 
move in the bin followed by an asterisk. 

As in Figure 7, whenever a moderate number (447) of moves is tabulated 
in this way, the distribution is noisy. For purposes of comparing two 
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PRINTOUT OF TIMING ANALYSIS 
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FIGURE 7 USE AND OUTPUT OF THE HISTOGRAM PROGRAM 
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distributions to see how they differ, It is desirable to smooth the 
distribution mechanically rather than by hand. As an option, the user 
may apply the smoothing function 


S 

i 



1 1+1 


which causes each bin in the smoothed function to contain half the 

counts in the same bin of the original function, C^, and one-quarter the 
counts in each neighboring bin, C. and C, . The impulse response of 
this smoothing function is shown in Figure 8, and the results of applying 
it to the distribution obtained in Figure 7 are shown in Figure 9. 
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(a) ORIGINAL IMPULSE 




FIGURE 8 IMPULSE RESPONSE OF SMOOTHING FUNCTION 









V TRACKING 


A. Three-Dimensional Compensatory Display 

In order to do tracking experiments, where the operator Is required 
to trace a path in space, a three-dimensional display is required* The 
operator must be able to look at the display and quickly assess his posi- ^ 
tion error along X, Y, and Z coordinates. 

A display suitable for this task consists of a movable circle and dot 
on an oscilloscope screen and a set of fixed, cross lines as shown in 
Figure 10. The circle and dot can quickly be visualized as an arrow 
pointing either into or out of the oscilloscope screen. With X and Y 
errors, both the circle and dot move left and right or up and down in 
unison; with Z errors the circle moves up and down with respect to the 
dot* A photograph of the actual display as seen on the scope screen is 
shown in Figure 11. 

The circuitry used to produce the display is shown in Figure 12. 
Command Inputs for the three coordinates are generated by the computer; 
response inputs for the three coordinates come from three position-sensing 
potentiometers that monitor the position of the manipulator end point; 
error outputs go to the computer for on-line analysis. The 200-Hz multi- 
vibrator drives two field-effect switching transistors, causing the 
deflection outputs to follow alternately the X and Y errors directly or 
the X and Y + Z errors with an AC perturbation generating the circle. 

This display can also be used to present two-dlmehsional , pursuit 
tracking tasks (the circle in the command position and the dot represent- 
ing the operator’s actual position). By commanding the circle to move 
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FIGURE 


FIGURE 10 TRACKING DISPLAY 
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11 PHOTOGRAPH OF TRACKING DISPLAY 
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FIGURE 12 DISPLAY GENERATION ELECTRONICS 





along a prescribed path, many different types of tasks can be simulated 
in the laboratory* 

B. Three-Dimensional Position Sensing 

In order to measure end-effector position, we designed a position 
sensor. Three of these sensors can be mounted in line with the three 
orthogonal axes (one sensor per axis). Each is attached to the end- 
effector via a control string. This is illustrated in Figure 13. 



FIGURE 13 MEASURING THREE COORDINATE POSITION OF THE END EFFECTOR 
(Only the Z-Axis Sensor is Shown) 


The sensor itself consists of a two-inch diameter pulley attached to 
the shaft of a ten-turn potentiometer. The control string is attached to 
this pulley, enabling the sensor to determine position over a range of 
approximately 6 feet. 
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The control string is provided with a constant return force by a 
direct-current motor acting as a negator spring. The motor is coupled 
to the pulley via a belt drive. 

In order to provide a velocity measurement in each of the three 
directions, a tachometer is mounted on the belt drive. The use of three 
velocity signals rather than six joint angles as input to the performance 
logger might considerably simplify the performance monitor software. 

In addition, each sensor has a suction-cup base and movable and 
fixed control string guides to facilitate mounting. The position and 
velocity sensor is illustrated in Figure 14. 
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TACHOMETER 


TORQUE MOTOR 




"CONTROL STRING 


MOVEABLE GUIDE 


PULLEY 


FIXED GUIDE 


SUCTION CUP BASE 


FIGURE 14 POSITION AND VELOCITY SENSOR 


VI TOUCH SENSING AND FEEDBACK SYSTEM 


A* Background 

During the period from 15 May to 15 November 1972, SRI proceeded 
with the development of a touch sensing and feedback system for the Space 
Nuclear Systems Office under Contract SNSN“63. The goal of this work was 
to design a system to provide, through a teleoperator, the touch informa^ 
tion normally used by man in directly manipulating objects with his hands* 
Although Contract SNSN-63 was prematurely halted because the Space Nuclear 
Systems Of f ice was closed, SRI had already completed the design of a set 
of sensorized tongs that can be fitted to the Navy arm,^ 

The previous work included (1) an overall design of a touch feedback 
system, (2) a selection of a sensor system based on bench- top tests, and 
(3) a selection of actuator systems based on bench-top tests, and (4) the 
design of a set of tongs based on results of 1 and 2. To date, we have 
built the pair of tongs and designed and almost completed the wrist and 
links to control the tongs, The results of this previous project and the 
adaptation of the system to the NASA-Ames arm are given in integral form 
for the first time in this report* 

B* Tactile Feedback Considerations 

Designs for a touch sensing system should consider (1) individual 
sensors and actuators, (2) the optimum encapsulation of the sensors in 
the end-effector, and (3) the arrangement of particular sensors on the 
tongs. The handgrip of the controller should serve the purposes of 
(1) providing a handle for transmitting six degrees of force to the arm 
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controller, (2) displaying tactile quantities to the hand, and (3) pro- 
viding one degree of freedom for opening and closing the end-effector. 

The integration of the important touch quantities into this control 
situation is shown in Figure 15. 

Based on our feasibility studies (surveys of sensor and actuator 
technologies^) and the requirement of fitting the system to the Navy end- 
effector^ and the MIT hand controller,"^ we have several recommendations 
for a tactile sensing system. 

Primarily, it should convey two types of touch information to the 
human operator. One of these is contact or touch with a high spatial 
resolution based on a matrix of sensors on the jaw surfaces and a corre- 
sponding matrix of position reproducing actuators on the palmar surfaces 
of the human finger and thumb. The other type is contact or touch with 
low resolution for relaying touch quantities from the exterior of the 
end-effector to the man*s hand as force reproducing actuators on the 
backs, sides, and tips of the human finger and thumb. 

The high resolution system should have at least a 3 x 6 matrix of 
sensor buttons that cover the end-effector gripping surfaces almost 
completely. The low resolution system should have two sensitive surfaces 
(uniformly sensitive to force over the entire surface) on each exterior 
surface of the tongs. 

Eventually, two commonly used prehension quantitites, force feedback 
and slippage feedback, should be included in a tactile sensing system. 

The role and implementation of these quantities is suggested in Figure 15 
and implementation of them into the system would give man nearly complete 
"feel” of the remote environment. 
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FIGURE 15 SENSING, CONTROL, AND DISPLAY OF TACTILE INFORMATION. Touch quantities commonly used by man 
in prehension are sensed on the end effector and anthropomorphically displayed on the hand controller. Dashed 
boxes are not to be considered in this study. 
























C. Selection of Touch Sensors 

After a survey of sensing mechanisms and bench tests was conducted, 
an optomechanical sensing scheme was selected. The system is simple, 
both electrically and mechanically, and it can be very rugged. It consists 
of a light-emitting diode (L.E.D.), an optical shutter, and a phototran- 
sistor, as shown in Figure 16. As the light vane is pushed down, a rec- 
tangular aperture opens permitting light to fall on the phototransistor. 

The amount of light permitted to enter is directly proportional to the 
deflection of the vane. 

The compact design of the shutter linked to a circular sensing pad 
on the surface of the end-effector Is shown in Figure 17. A compliant 
washer (or spring) between the body of the tongs and the sense pad con- 
verts external force to deflection. By varying the compliance of the 
washer, the sensor can be adjusted to have full-scale output for applied 
weights from two grams to many kilograms. 

D. Selection of Actuators 

The survey of actuators for providing tactile stimulation to the 
operator’s hand clearly suggested that rotary solenoids were best for 
this application. A bench-top design with rotary actuators and push-button 
stimulators is currently being used to develop tactile stimulators for 
both the high- and low-resolution systems. Another viable possibility 
being considered is that of using miniature torque motors with a rotary- 
to-llnear displacement converter. A particular problem at this stage is 
the design of the push-rod mechanism that couples actuators and push- 
buttons. 


27 



0 



FIGURE 16 EXPLODED DRAWING OF SHUTTER 



FORCE 



FIGURE 17 CROSS SECTION OF SENSOR WITH COMPACT SHUTTER 


E. Hand Design 

To experiment with and evaluate touch sensors in a working control 
system, an end-effector has been designed and partially constructed. 

The end-effector consists of drive motor, housing, and links to support 
and control the sensorized tongs designed for the Navy hand.^ Prelim- 
inary measurements indicate that the sustained gripping force is 20 pounds 
and that the time to close the hand from an open position is 0.5 second. 
The speed and force characteristics were chosen to match the tongs to 
the NASA-Ames arm. Even higher opening and closing speeds may be neces- 
sary to allow the operator to effectively sense and control touch infor- 
mation fed back to him. 

During the next quarter the hand should be completed, and we will 
make available a full description complete with a series of photos and 
sketches. 
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VII INFRARED POSITION SENSOR 


During this quarter we have built a set of circuits to drive the 
sensor described in Appendix A of Reference 1 for bench-top testing. 

The circuitry is undergoing change and will be described at a later 
date. Experiments with the system include determination of (1) optimum 
focus and maximum strength of the light beams and (2) optimum focus and 
maximum receptivity of the phototransi stor ”eyes.” 

In order to make these measurements, we mounted the infrared posi- 
tion sensor on one edge of an XY plotter table. The movable carriage 
used a raster scan to search the area in front of the sensor. By mount- 
ing a light sensor or emitter on the movable carriage and controlling 
the up-down position of the pen from the detector output, we plotted 
field strengths. 

An example of such a field-strength plot is given in Figure 18. 

Here a phototransistor with a small window (1-mm diameter) was swept 
slowly through the field on a raster scan. The period of the vertical 
sweep is about 10 seconds, and that of the horizontal sweep is about 10 
minutes. The plot of Figure 18 reveals that two side lobes (not designed 
into the light-emitting system) are present at the top of the picture, 
and that the image plane of the light emitters is 2.2 inches in front of 
the sensor housing. Using these procedures it should be possible to map 
and control the receptive areas of the position sensor for best use in 
manipulation tasks. 
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